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ABSTRACT 

This report on an evaluation of Ohio *^tate-f unded 
career enhancement pilot programs cover six school districts. Four 
kinds of programs are covered: (1) career ladder — teachers are 
promoted through a series of steps based on different levels of 
competence and responsibility; (2) career option — teachers take on 
and are compensated for defined responsibilities related to the 
classroom, but increases in pay last only as long as participation 
does; (3) shared governance — school-level decision making is shared 
by administrators, teachers, parents, and community members; and (A) 
merit pay — bonuses or pay increases are awarded to teachers who have 
performed particularly well. In five of the six districts there is a 
peer review process that uses teachers rather than administrators to 
evaluate and coach one another. Study findings indicated that 
classroom changes resulting from career enhancement activities 
increased students* opportunities to learn, and that the most 
promising career enhancement strategies programs were peer review and 
targeted staff development. Recommendations based on the findings 
include: (1) allow all districts who want career enhancement to 
compete for grant funds; (2) incorporate peer review and staff 
development targeting individual districts' needs into statewide 
professional development initiatives; and (3) address the design and 
implementation flaws of the merit pay pilot program. Survey questions 
are appended. (Contains 41 references.) (ND) 
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The Legislative Office of Education Oversight (LOEO) serves as staff to the Legislative 
Committee on Education Oversight. Created by the General Assembly in 1989, the Office 
evaluates education-related activities funded wholly or in part by the state of Ohio. 

Since 1987, the career enhancement line item has allocated funds for pilot programs in six 
school districts. As the line item's name, "Career Enhancement," implies, these programs are 
intended to enhance the career of teaching, and by doing so, improve the quality of educational 
experiences and performances of students. This report examines these state-funded career ladder 
and peer review pilot programs for teachers. Conclusions and recommendations in this report 
are those of the LOEO staff and do not necessarily reflect the views of the Committee or its 
members. 



SUMMARY 



OfflO’S CAREER ENHANCEMENT PILOT PROJECTS 



The Legislative Office of Education Oversight (LOEO) examined 
state-junded career enhancement pilot programs and found them to 
be an effective means of improving the quality of teachers in 
participating districts. Most problems with career enhancement 
programs were caused by conflicts between individuals or difficulties 
with program implementation, rather than program design. 



Since 1987, the career enhancement line item in the state 
budget has allocated funds for pilot projects in six school districts. 
As the line item's name, "Career Enhancement," implies, these 
programs are intended to enhance the career of teaching, and by 
doing so, improve the quality of educational experiences and 
performances of students. Traditionally, teachers receive increases in 
pay based on total years of service and completion of college 
coursework. Outstanding performance in the classroom is minimally 
recognized, and most opportunities for advancement require teachers 
to leave the classroom permanently. 



State funds assisted six districts with 
carsar snhancamsnt pilot projects, 
each of which developed its own 

program. 



LOEO evaluated the pilot programs in: Arlington Local, Berea 
City, Cincinnati Qty, Qeveland City, Columbus City, and Toledo 
City school districts. Each district was required to match the 
designated state funds with local funds. Amounts distributed to 
individual districts varied. In fiscal year 1993, the districts received 
amounts ranging from $33,000 to $281,850. 



Each of the districts developed its own program. As a result, 
money from the career enhancement line item has funded several 
kinds of pilot programs: career ladder, career option, merit pay, and 
shared governance. A career ladder program promotes participating 
teachers through a series of steps based on different levels of 
competence and responsibility. A career option program allows 
te.achers to take on defined responsibilities related to the classroom; 
teachers are compensated for participating, but increases in pay last 
only as long as the participation does. Shared governance activities 
promote school-level decision making, shared by administrators, 
teachers, parents, and community members. A merit pay system 
awards bonuses or pay increases to teachers who have performed 
particularly well. 

In five of the six districts there is a peer review process that 
uses teachers, rather than administrators, to evaluate and coach one 
another. Peer review processes are applied to all new teachers, as well 
as to assist experienced teachers identified as needing improvement. 
Career ladder programs use peer review to make decisions about 
advancement. 
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LOEO concludsd that classroom 
changos rosulting from careor 
anhancomant activitlas Incraasad 
studants* opportunitias to learn. 



Tha most promising strategies used 
by career enhancement programs 
wore peer review and targeted staff 
development 



Career enhancement programs are intended to produce changes in 
teachers and thus bring about changes in student learning. LOEO concluded 
that career enhancement activities in the classroom increased students' 
opportunities to learn. This study found that: 

^ Career enhancement pilot programs have demonstrated 
success by benefiting participating teachers and their 
students. 

> Advantages brought by these programs include developing 
new skills for teachers to use in the classroom, providing 
the best teachers an incentive to remain in the district, and 
compelling ineffective teachers, including some 
experienced teachers with tenure, to leave the profession. 
High-performing teachers were rewarded with increased 
salaries, and in most cases, with increased responsibility, 
status, or respect. Teachers assumed leadership roles that 
were once solely administrative, such as evaluation and 
personnel decisions. Education reform efforts were 
enhanced by this professionalization of teachers' careers, 
which included increased accountability and collaboration 
among teachers, administrators, and the teachers' uniotts. 

> Disadvantages resulted from the way the program was 
implemented in a specific district rather Aan inherent 
problems in the program itself. Disadvantages, such as 
personality clashes, administrative problems, and jealousy 
resulted from the interaction of individuals. 

> The most promising strategiv’s used by career 
enhancement programs were peer review and targeted 
staff development. The peer review process was 
identified as effectively removing the worst teachers from 
the classroom. Peer review provided opportunities for 
teachers to evaluate and assist one another. Staff 
development targeted to specific needs of a district's 
teachers allowed them to acquire new skills to help their 
students learn. 

> The merit pay program gave its better teachers an 
incentive to stay in the district by rewarding them for 
good evaluation scores with an increase in pay. However, 
the merit pay pilot program did not include' activities to 
improve the skills of participants or a way to remove the 
least effective teachers. Its criteria for pay increases were 
subjective, and results of participants' evaluations did not 
remain confidential. These attributes detracted from the 
value of this pilot program. 
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Based on the findings of this study, LX)EO recommends that: 



LOEO racommanda that tha Ganaral 
Aaaambly allow diatricta that wiah to 
davaiop caraar anhancamant 
programa to apply for two*yaar 
davalopmant or atart«up funda. 



^ The General Assembly allow all districts who want career 
enhancement programs to compete for grant funds. The 
success of six different programs has shown that career 
enhancement can be advantageous to participating 
districts. There is no need to continue testing the 
concepts of career enhancement in pilot projects. Districts 
with existing career enhancement programs and districts 
that wish to develop new ones should compete for line 
item funds. 

ODE would develop guidelines for distributing funds, 
based on Onancial need and the potential of proposed 
programs to affect both students and teachers. Districts 
would be strongly encouraged to include peer review and 
targeted staff development components in their plans. In 
addition, each plan would be required to include regular 
self-evaluation of both the program's implementation and 
results. 

► The Ohio Department of Education incorporate peer 
review and staff development targeting individual districts' 
needs into any statewide professional development plan it 
initiates. Those activities should continue where they 
exist, and be incorporated into future professional 
development activities of other districts. 

► School districts that wish to use merit pay programs 
address the design and implementation flaws of the pilot 
program. Redesign should ensure that the program's 
evaluation and reward process is objective and there are 
program activities to help teachers improve. 
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CHAPTER I 
INTRODUCTION 

Since 1987, the career enhancement line item has allocated funds 
for pilot programs in six school districts. As the line item’s name, 
"Career Enhancement, " implies, these programs are intended to 
enhance the career of teaching and by doing so, improve the 
quality of educational e:q>eriences and performances of students. 
This report examines these state-funded career ladder and peer 
review pilot programs for teachers. 



LOEO examined state-funded career 
enhancement pilot programs, and overall found 
them to be an effective means of improving the 
quality of teachers in participating districts. In 
general, the programs provided incentives for the 
best teachers to remain in the districts' 
classrooms and increased the skills of 
participating teachers. Peer review processes, in 
which teachers are evaluated and assisted by 
outstanding experienced teachers instead of 
administrators, resulted in the removal of the 
least effective teachers from the classroom. 
When there were problems with career 
enhancement, they were caused by conflicts 
between individuals or difficulties with the 
implementation of a program, rather than its 
design. 

Each of the six districts developed its own 
program. As a result, money from the career 
enhancement line item has funded several kinds 
of programs: career ladder, career option, merit 
pay, and shared governance. In five of the six 
districts there is a peer review process; the sixth 

SCOPE AND 

Amended Substitute House Bill 152 
required LOEO to evaluate career ladder and 
peer review programs. The act states: 

The Legislative Office of Educa- 
tion Oversight shall evaluate the 
use of peer review and career 
ladder programs in improving 
instructional performance and shall 
report to the chairman of the 
Legislative Committee on Educa- 



district's merit pay program does not include peer 
review activities. 

The Ohio Administrative Code requires 
the Division of School Finance of the Ohio 
Department of Education (ODE) to conduct an 
annual onsite review to determine program 
effectiveness. This rule was adopted in 1990. 
However, according to staff at ODE, no onsite 
reviews have been conducted. Originally, ODE 
planned to evaluate career enhancement 
programs as part of school district reviews that 
occur once every five years. The school district 
review process was discontinued in 1991, and to 
date ODE has not conducted a systematic 
evaluation of the career enhancement programs. 

All programs have a governing board at 
the local level responsible for annual evaluations, 
and all districts have evaluated their programs 
locally in some fashion. The structure and 
thoroughness of these evaluations varies, 
however. 



METHODS 

tion Oversight by September 1, 

1994, concerning the relative 
advantages and disadvantages in 
implementing such programs. 

Because the line item now called "career ladders" 
has funded several types of career enhancement 
programs, LOEO's study included all programs 
funded by the line item. 



Scope 



LOEO reviewed programs in the six 
school districts currently receiving state funds 
from the Career Ladders line item. Although 
other districts in the state may carry out similar 
activities, LOEO did not review their programs. 

This report addresses three issues: 

► How did the use of state-funded 
career enhancement programs affect 
instructional performance? 



► What were the advantages and 
disadvantages of implementing these 
programs? 

► What role did peer review play in 
career enhancement programs? 

LOEO looked for changes in teachers' 
activities, and resulting changes in the classroom 
for both teachers and students. LOEO identified 
advantages and disadvantages occurring across 
the different types of programs. Included within 
our examination was the contribution of each 
program to school reform efforts. As we 
answered the first two questions, LOEO explored 
advantages and disadvantages of peer review, 



and sought links between it and changes in 
instructional performance. 

Methods 

LOEO staff reviewed the research 
literature concerning career ladders, career 
options, merit pay, shared governance, peer 
review, and related activities. (Bibliography is 
found in Appendix A.) We made several 
computer network inquiries and contacted 
participants in career ladder programs of several 
states. LOEO staff interviewed Ohio Department 
of Education personnel. 

To collect information for this study, 
LOEO interviewed a total of 21 local 
superintendents, other administrators, and union 
officials in participating districts. We also 
surveyed principals and teachers in those 
districts. (Survey questions are found in 
Appendix B.) 

We included nonparticipants in our sample 
to learn their opinions about the effects of career 
enhancement programs on the school, the district, 
and students other than those in participants’ 
classrooms. LOEO anticipated that 68 surveys 
would be returned; we received 50. Of those 50, 
only six came from nonparticipants. 
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CHAPTER II 

DESCRIPTION OF CAREER ENHANCEMENT 



This chapter describes the history of Ohio's career enhancement activities. 
Since a 1984 education reform initiative first recommended that career 
enhancement and peer review programs be established in Ohio, the state 
has funded programs in six districts. These programs were based on 
education research regarding the use of career enhancement to 
professionalize teaching and improve schools. A brief overview of each 
district's program illustrates how the district designed its own program to 
meet its specific needs. 



HISTORY 



Traditionally, a classroom teacher's role 
changed little throughout her career. Teachers 
assumed leadership roles in their own schools 
and at the district level by participating in union 
activity, or serving as department chairs and 
members of advisory and governing committees. 
Professional advancement usually required 
leaving the classroom to assume administrative 
duties. 

Although the state has a minimum salary 
schedule for teachers, each district determines 
how far above the minimum its salary schedule 
will be. In most districts, a teacher's salary 
depends more on total years of employment and 
education than achievement or performance in 
the classroom. To earn more, a teacher must 
either leave that district, or leave the classroom 
altogether. Thus, some districts often lose their 
best teachers. 

In the early 1980s, education research 
suggested that changes in the organization of 
schools and the roles of teachers would improve 
schooling. Furthermore, research found that the 
most effective initiatives were those developed 
locally with input from teachers. Teachers were 
to have more say in decisions that affected the 
classroom and more opportunities to develop 
their skills and knowledge. A teacher in this 
setting would be able to advance professionally 



without leaving the classroom. For example, an 
experienced teacher might be paired with a new 
teacher and spend some school time in the new 
teacher's classroom, observing and helping. Or, 
the experienced teacher could be encouraged to 
create a series of inservice staff development 
sessions for all the teachers in the building, and 
be compensated for doing so. 

In 1984, the State Board of Education 
adopted a reform effort, the Master Plan of 
Excellence, which recommended establishing 
career enhancement and peer review. A grant 
from the U.S. Department of Education allowed 
the Ohio Department of Education (ODE) and 
Miami University to conduct a feasibility study, 
which was completed in June 1987. 

The legislature provided funding for pilot 
projects in Toledo and Columbus during the 
1987-1989 biennium. In June 1989, ODE was 
authorized to develop and submit a plan to the 
General Assembly by December 1990 for 
phasing in career enhancement programs 
statewide. Between 1987 and 1991, each state 
budget included a line item that identified 
districts that would receive funds for their career 
enhancement activities. 

In the 1989-1991 and 1991-1993 biennia, 
funds were designated to assist seven districts 



with career enhancement pilot programs: 
Arlington Local, Berea City, Cincinnati City, 
Cleveland City, Columbus City, Felicity-Franklin 
Local, and Toledo City. Felicity-Franklin did 
not accept these funds. Because it was a small 
rural district, administrators felt that it did not 
have resources to replace state funding when the 
pilot project ended! Each district was required to 
match the designated state funds with local 
funds. Amounts distributed to individual 
districts varied. In fiscal year 1993, the six 
districts received amounts ranging from $33,000 
to $281,850. 

Amended Substitute House Bill 152 
appropriated approximately $1.1 million for 

DESCRIPTION 

The Ohio Department of Education 
submitted Ohio's Plan for Phasing in Career 
Ladders to the General Assembly in December 
1990. This study suggested that the state could 
most effectively promote career enhancement by 
providing assistance for districts to develop local 
incentive programs. The plan describes existing 
programs of Toledo, Columbus, and Arlington, 
and suggests that to create a career enhancement 
program a district could follow one of these 
models or create something completely different. 

It used studies of other career enhancement 
programs to develop 11 components districts 
should include when creating any new career 
enhancement plan. ODE would approve any 
new district career enhancement programs prior 
to funding. 

Ohio’s Plan for Phasing in Career Ladders 
states, "the mission of Ohio's career enhancement 



career ladder programs for each year of the 
1993-1995 biennium. However, language allo- 
cating these funds differed from that of its 
predecessors. The name of the line item changed 
from "Career Enhancement Programs" to "Career 
Ladders." Am. Sub. H.B. 152 did not sp>ecify 
districts to receive funds, but said that ODE 
could distribute funds to districts with existing 
programs that had used previous grants 
effectively. According to ODE, Hscal year 1994 
funds were distributed upon review of program 
descriptions and available local evaluations. The 
criterion used to determine if a district received 
funds was whether the reported program 
activities conformed to a district’s stated 
intentions. 

PROGRAMS 

programs is to improve the quality of educational 
experiences and performances of students, and to 
enhance the career of teaching." It describes 
three models of career enhancement programs: 
career ladder, career option, and merit pay. Each 
district's program was designed to meet its 
unique needs. Location, size, and composition 
of each district affected the development of its 
particular career enhancement activities. For 
example, the large number of teachers in the 
Cleveland City School District made the cost of 
including more than a few teachers in a career 
ladder program prohibitively high; that district 
chose to use career enhancement funds for its 
shared governance program. 

Exhibit I illustrates the kind of program 
each district chose to meet its needs. 




EXfflBIT I 

OHIO’S CAREER ENHANCEMENT PILOT PROGRAMS 







Career Ladder 


Career Option 


Shared Governance 


Merit Pay 


Cinciimati 


Berea 


Cleveland 


Arlington 


Toledo 


Columbus 







Career Ladder 

A career ladder program promotes 
teachers through a series of steps based on 
different levels of competence and responsibility. 
As career ladder teachers' responsibilities 
increase beyond those of nonparticipants, they 
are promoted from step to step, and their pay 
levels rise above those of the district salary 
schedule. 

Toledo. Toledo uses a career ladder 
program to provide incentives to recruit and 
retain quality teachers. The career ladder 
contains six phases. During the first two phases, 
interested teachers undergo intense evaluation, 
including being observed and evaluated by their 
peers, while pe'-forming their regular teaching 
duties. In the tiiird phase, qualifying teachers 
step onto the first of four "rungs" of the ladder. 

Those teachers who qualify to "climb the 
ladder" choose from among five career tracks: 
(1) professional development, (2) curriculum, (3) 
community leadership, (4) educational technol- 
ogy, or (5) performance. They then are assigned 
activities or projects within their chosen tracks. 
These activities are carried out in addition to 
their regular full-time classroom duties. For 
example, a teacher choosing the curriculum 
track might direct a textbook selection 
committee, develop a preschool program, direct 
a curriculum study committee, or research 
curriculum innovations. A teacher focusing on 
educational technology might develop compe- 
tency testing, improve testing by developing 
alternative ways of measuring student progress, 



or preview new software and provide information 
to all staff. 

Toledo is an urban district, serving nearly 
38,000 students. After three years of study, 
Toledo began its program in 1987. Aiming to 
increase the professionalization of teaching 
within the district, union representatives and 
administrators used components of a peer review 
intem/intervention plan already in place to create 
a career ladder. Total state dollars appropriated 
for fiscal year 1993 were $281,850; these funded 
salary increases for the program's 57 career 
ladder teachers. The district provided matching 
funds of $592,481 by funding the 
intem/intervention program. 

Cincinnati To attain its goal of providing 
teacher support and developing curriculum and 
leadership resources, Cincinnati's Career in 
Teaching Program provides opportunities for 
teachers to move through four sequential levels: 
Intern, Resident, Career, and Lead Teacher. 
Interns are first-year teachers; residents have less 
than eight years of experience and no tenure; and 
career teachers are those with tenure. Lead 
teachers may conduct peer appraisals of new 
teachers, mentor other teachers, facilitate new 
programs, head newly created curriculum and 
specialist councils, or serve as department chairs 
or elementary level leaders. 

Cincinnati City School District is a large 
urban district serving a diverse student 
population of approximately 50,000. In 1985, the 
Cincinnati Federation of Teachers proposed an 
intern program to assist first-year teachers. This 




idea was adopted in 1986. During 1988 and 
1989, a joint union-administration committee 
drafted an additional, more elaborate, four-stage 
care>-f ladder plan. This plan took effect in 
September 1991. In 1993, state contributions to 
the Career in Teaching Program totaled 
$234,900. The district matched the state 
contribution with money set aside by a teachers' 
contract settlement. State funds were spent for 
pay supplements for lead teachers. 

Career Option 

A career option program allows teachers 
to take on defined responsibilities related to the 
classroom. Teachers are compensated for 
participating, but increases in pay last only as 
long as the participation does. 

Berea. Berea's career option program 
compensates experienced teachers for mentoring 
teachers with three years or less experience in 
that district. Currently, 24 teachers serve as 
mentors. Tenured teachers who are identified as 
needing intervention also receive "clinical 
supervision." 

Berea City School District serves 
approximately 8,000 students in a suburban 
setting. In the early 1980s, educators in the 
district realized that in the next five to ten years 
they would be losing many teachen. to 
retirement. To improve chances to recruit and 
retain quality teachers, a development committee 
of union members and administrators began to 
create a program in 1986. After researching and 
examining other career enhancement plans, they 
developed a career option program. In fiscal 
year 1993, the state contributed $45,789 to 
Berea's Consulting Teacher program, which the 
district matched with local funds. Money for the 
program was spent on supplemental pay for 24 
mentors and for substimtes to cover their 
classrooms when release time was necessary. 

Columbus. Columbus' career option 
program {)rovides several different opportunities 
for experienced teachers. Currently, state funds 



directly support one of the components, the Peer 
Assistance and Review (PA.») program. 
Classroom teachers are selected to become PAR 
consultants. They review and assist two groups 
of teachers--newly hired teachers and ineffective 
experienced teachers. Consultants observe or 
meet with each assisted teacher at least 20 times 
a year. PAR consultants may remain in this full- 
time position for a maximum of three years, and 
then must return to classroom teaching. Other 
options, currently supported by local funds, 
include becoming a competency-based education 
(CBE) consulting teacher, participating on a 
reform panel, or serving on the curriculum 
coordinating council. 

The Columbus City School District serves 
a large, diverse student population of about 
60,000. In 1987, the Columbus City School 
District merged several preexisting programs to 
create a career option program. Expenditures 
from state contributions to the Columbus career 
option program were $187,850 in fiscal year 
1993. Most of the state funds paid salaries of 
PAR consultants. The district matched the funds 
with support of the CBE consulting teacher 
option. 

Shared Governance 

Shared governance activities promote 
school-level decision making, shared by 
administrators, teachers, parents, and community 
members. 

Cleveland. Career Enhancement funds 
were used to support development of self- 
governance methods in the district. During the 
1991 and 1992 school years, funds were used to 
support training in decision-making skills at 13 
pilot sites and included expenditures for small 
training stipends for teachers, administrators, 
parents, and community members, as well as 
substitute coverage for teachers and 
administrators. Funds were also used to hire 
consultants to provide the training, and for the 
purchase of necessary materials and supplies. 
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aeveland is Ohio's largest district, 
serving approximately 72,000 students. In 1990, 
the district received a commitment from both the 
Cleveland Board of Education and the Qeveland 
Teachers Union to pursue the development of a 
shared-governance and joint-decision-making 
model designed to benefit students, enhance 
community participation, and to increase the 
gratification professionals receive. In fiscal year 
1993, Qeveland received $154,845 in state 
career enhancement funds, which they matched 
with $1.25 million in funds for peer review and 
$400,000 in staff development funds. 

Merit Pay 

A merit pay system awards bonuses or 
pay increases to teachers who have performed 
particularly well. 

Arlington. In Arlington Local School 
District's merit pay program, teachers' pay levels 
are adjusted according to their scores on two 
evaluations conducted by their principals. 
Principals evaluate each participating teacher in 
the classroom twice a year and forward 
completed evaluation forms to the 
superintendent. The superintendent rank-orders 
scores and then uses the rankings to decide pay 
levels. 



Arlington Local School District is a rural 
district in Hancock County, serving approxi- 
mately 670 students. Because of the small size 
of the two schools in the district, most teachers 
are aware of each others' activities and opinions. 
Arlington developed a merit pay program to 
address the problem that its best teachers often 
leave to take employment in districts capable of 
offering higher salaries. School board members 
working in businesses that use merit pay 
proposed a merit pay program for the district. 
Members of the district's board of education 
worked with the superintendent and the teachers' 



union to design the merit pay program. Several 
teachers developed the forms used to evaluate 
teachers. In fiscal year 1993, the district matched 
each state dollar with four dollars of local funds, 
and spent a total of $147,983 to provide pay 
increases to all of its teachers. 

Peer Review 

Ohio's Plan for Phasing in Career Ladders 
describes the peer review process as using 
"teachers who have demonstrated mastery of the 
skills and the knowledge which is to be 
evaluated." A peer reviewer's duties may include 
determining a teacher's eligibility to participate in 
a program, identifying specific skills a teacher 
needs to improve, recommending continued 
employment of a beginning teacher, or providing 
one-to-one assistance to new and experienced 
teachers. Peer review uses teachers, rather than 

administrators, to evaluate and coach one 
another. Several programs have a particularly 
strong focus on peer review and assistance of 
new teachers. However, peer review processes 
are not applied only to new teachers. They are 
instrumental in decisions about advances on 
career ladders, and are used to assist experienced 
teachers identified as needing improvement. 

Peer review is a component of all four 
state-funded career option and career ladder 
programs. A separate peer review program 
exists in one of those districts and the district 
that uses shared governance. There is no peer 
review process in the district using a merit pay 
program. 

Reform Efforts and Career Enhancement 

Ohio's career enhancement initiative 
included the features identified by research and 
other states' programs as leading to successful 
reform efforts; 
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► Changes result from local efforts. 

► Design and implementation of new 
ideas involve teachers as participants. 

► Any redesign of school structure or 
process acknowledges that school is a 
workplace for teachers, and supports 
ongoing development of teachers. 

Approximately three fourths of respond- 
ents to LOEO's survey saw their districts' career 
enhancement programs as being the embodiment 
of school reform concepts. A teacher from a 
career ladder program asserted: 



Our...program is at the heart of om 
reform and restructuring efforts. 
The actual structure of our organ- 
izational chart has changed. We 
no longer have supervisors; those 
responsibilities have been taken 
over by lead teachers. Any com- 
mittee that is looking at any kind 
of educational change in our 
system has a large number of lead 
teachers involved in [it]. 
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CHAPTER III 
PROGRAM EFFECTS 



This chapter discusses the effects of career enhancement programs on 
teachers and students, and the advantages and disadvantages reported from 
each kind of program. It also examines the contributions that the peer 
review process made to career enhancement programs. The findings are 
reported in terms of advantages and disadvantages. 



Each section of this chapter begins with a 
chart to summarize the overlapping effects and 
activities of the six different programs. The 
chart presents the information LOEO collected 
during interviews and surveys, and acknowledges 



the programs from which responses came. 
Further information about each item in the chart 
is contained in the subsequent text. The charts 
are consolidated in Exhibit 6. 



ADVANTAGES 



One of the premises of career enhancement programs is that changes in teachers bring about 
changes in student learning. Classroom changes resulting from career enhancement activities increased 
students' opportunities to learn. These changes included increasing the quality of teachers by retaining 
the best and releasing the least effective teachers; improving teachers’ methods of instruction; and 
changing the focus of classroom activities. Increased opportunities to learn are reflected in improved 
student attitudes, better classroom test scores, and more frequent extracurricular activities. 

EXHIBIT 2 

OVERALL ADVANTAGES OF CAREER ENHANCEMENT PROGRAMS 
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Students had increased opportunities to 
learn, because the programs caused 
improvement in overall teacher quality. 
District administrators and survey respondents 
from all districts asserted that because of the 
career enhancement program, the overall quality 
of their teachers had imp? 'ed. With the 
exception of merit pay, these programs had the 
effect of removing the worst teachers from the 
classroom. 

In five districts, the worst teachers left 
because of the peer review process. According 
to LOEO survey respondents, peer review 
activities resulted in nonrenewal of contracts of 
ineffective teachers, including some with tenure, 
in five districts receiving career enhancement 
grants. According to an administrator, about two 
percent of all teachers being monitored and 
assisted never reach a skill level that will allow 
effective teaching, in spite of ample assistance. 
Their removal results in students no longer 
wasting years in their classrooms. A consulting 
teacher stated that, although recommending 
nonrenewal is difficult, it allows some new 
interns "to see that teaching may not be a wise 
career choice and they are free to make a new 
choice, before they have invested many years on 
the job, and before their poor teaching has 
affected many children." 

Career enhancement programs ofller the 
best teachers an incentive to remain in 
teaching and in the district The best teachers 
are most able to motivate students and help them 
learn. Teachers and school administrators 
reported that program incentives included more 
pay, advancement opportunities, changes in 
responsibility, and professionalization of 
teaching. 

Participants found salary increases a 
valuable form of recognition, as well as a 
financial incentive to work hard. One participant 
stated, "The biggest advantage has been the pay 



increase. If you have done well, you feel, let's 
say, recognized or appreciated." 

Survey respondents discussed the 
advantages of having advancement 
opportunities. The possibility of an upward 
career move for the best teachers, said one 
principal, "helps attract and retain good teachers, 
keeping them in teaching as opposed to 
administration, and keeping them in the 
[district's] schools as opposed to another 
profession." 

Participants, both peer reviewers and their 
mentees, noted a change in their perspectives of 
their classrooms and districts. One stated, "It 
rejuvenated me." Another said, "I see a bigger 
picture of the district, and of teaching. Working 
as a consultant gives a much broader picture than 
from just a classroom or school." 

Program activities, such as staff 
tniining sessions, observation of other 
teachers, and peer review, improved the skills 
and techniques of participating teachers. 
These improved skills allowed teachers to 
increase student learning. Although tangible 
evidence of student learning is often difficult to 
produce, participants cited increased student test 
scores on both classroom and norm-referenced 
tests. For example, one recipient of consulting 
teacher services clearly linked changes he had 
made in his teaching to changes in student 
performance. He stated: 

I am doing things very differently 
now. I think the students learn 
more. I see a great improvement in 
the classroom test grades. 1 used 
the same tests several years in a 
row, and saw improvement in these 
scores. I just got back national 
[subject] test scores and they were 
great. 
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PROGRAM ADVANTAGES THAT AFFECT STUDENTS 





Career Option 


.f \ 

'U* O' 


Shared Governance 


iji 

J;' y ' : 

i’ti 


Teachers rewarded for professional growth 


X 




X 




Participation in university workshops or inservice sessions provided by 
the district 


X 


X - 


X 


- " / 


Student activities to change attitudes 




; ^ 


X 




Peer assistance in classroom management 


X 




X* 




Increased resources 


X 


'1 


X 


; ' 


Increased sharing of ideas 


X 




X 




Observing other teachers 


X 




X* 




Self examination 


X 






: 


Change in curriculum or its focus 


X 




X 




Increased extracurricular activities 


X 




X 


X ^ 



These were reported as pari of the peer review program supported by matching funds. 



Expansion of teachers' perspectives, skills, 
and knowledge were cited frequently as effects 
of these programs. Each program promoted this 
professional growth in its own way. 

For example, teachers in one program are 
rewarded for taking initiative to learn new skills. 
One teacher said, "The teachers at the lop of the 
pay scale are now less likely to vegetate there. 
Instead, they are attending meetings, and are 
more likely to grow professionally." 

Other career enhancement programs 
provide university workshops or require 
participants to prepare inservice sessions. This 
training helped participating teachers to diversify 
their leaching methods. Teachers reported that 



their students now receive more individual 
instruction; cooperative and group learning 
activities are more frequent; and students who 
might once have passively listened to a teacher 
lecture, and understood little, are engaged in 
hands-on activities that allow them to learn. 

Teachers described program activities 
that fostered attitude changes in students, such 
as increased motivation and better discipline. 
For example, shared governance teams developed 
programs that encourage students to develop 
contracts to alter their behavior and to engage in 
dispute mediation. 

In the peer review process, experienced 
teachers counsel new teachers intensely. A 
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consulting teacher stated that she has observed or 
counseled each Hrst-year teacher at least 30 
times this year. As one participant stated: 

Consulting teachers offer interns 
assistance and guidance, so that 
they are not thrown into a new 
situation with no help. New 
teachers are better able to concen- 
trate on instruction, because they 
are getting assistance. They don't 
have to reinvent the wheel. They 
can ask for help and get tips and 
strategies and techniques for 
handling discipline problems. 

Participants in the peer review process 
pointed out that being paired with an 
experienced teacher provided new teachers 
with support and an easily accessible source of 
information. One respondent said, "By being 
paired with someone to review their classroom 
management techniques and teaching strategies, 
the new teachers could use their best techniques 
and not waste a lot of student and teacher time 
trying to use unsuccessful ones." 

Improvement in skills relating to 
classroom management resulted from peer 
review's consulting teacher/intem relationship. 
For example, a consulting teacher reported 
working with a teacher whose classroom was 
total chaos. The consulting teacher suggested 
providing "so much structure, so many activities, 
that the kids have no choice but to participate. 
The rules became 'you must be here by 10:00, 
your journal entry must be complete by 10:05, 
your reading must be finished by 10:20, etc.'" 
Lesson activities moved quickly, and grade 
penalties for not meeting the requirements were 
established. Within a short time, the classroom 
teacher had the skills to maintain order, and the 
discipline allowed subject matter not only to be 
presented, but to be learned. 

Survey respondents described an 
increased availability of resources to individual 
teachers and their students. Program participants 



spent time procuring or creating equipment and 
written materials for sc’ence lab classes, 
literature reflecting a variety of cultures, and 
assistance with designing an advanced placement 
English course. Participants depicted this 
increased availability as particularly beneficial to 
new teachers. 

Participating teachers reported an increase 
in sharing ideas and conferring. One consulting 
teacher described working with a new teacher. 
The consulting teacher shared ideas on classroom 
management, the intern shared how he used 
computers in the classroom, and both teachers 
changed their classroom practices. 

Participants related changing their own 
teaching after observing that of others. One 
consulting teacher described a new teacher who 
had difficulty ensuring that all students were 
included in oral questioning. She suggested that 
the new teacher keep notecards for all students, 
containing records of her frequency of question- 
ing them, and their success in answering. When 
she reviewed her own classroom techniques, she 
found them to be only moderately successful, 
and therefore followed her own advice and began 
to keep notecards in her classroom. 

Reflection and self-examination of a 
teacher's own work were products of career 
enhancement programs. One experienced teacher 
described an unsuccessful attempt to enter his 
district's career ladder program, and the changes 
the process caused in his teaching. Throughout 
the year in which he was evaluated, eight 
different teachers each observed his classroom 
three times. After each of the 24 observations, 
he thought about his activities, and how he might 
improve them. Although he was not placed in a 
higher position, he feels the program is 
responsible for increasing his skills and 
development as a teacher. 
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Survey respondents noted that career 
enhancement programs facilitated district-wide 
changes in the focus of classroom activities. For 
example, in one district, program participants 
changed the focus of the literature curriculum, 
and students now study literature with a multi- 
cultural emphasis. In another district, consulting 
teachers helped develop consistency in curricu- 
lum focus among the district's classrooms. 

Administrators and participants reported 
that career enhancement programs increase 
extracurricular activities offered to students. 
For example, career ladder teachers who focus 
on community relations offer after-school 



activities for both students and parents. Merit 
pay teachers are rated on their participation in 
parent organizations and for advising 
extracurricular activities, so more teachers now 
volunteer. The shared governance program 
encourages community involvement in its 
schools' management. Not only teachers and 
administrators, but parents and community 
members, appear on the governing team of each 
participating school in that district. Activities to 
assist high school students prepare for statewide 
proficiency testing were reported by admin- 
istrators and survey participants. Respondents 
attributed increased success on proficiency tests 
to these activities. 
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Program activities also brought 
changes for teachers that did not necessarily 
affect students. These programs provided 
opportunities for advancement that changed 
teacher and principal responsibilities. 

For teachers, some of these new 
responsibilities were activities that formerly 
were carried out only by administrators— 



evaluation of teachers, for example. Additional 
responsibilities— for example, curriculum devel- 
opment or department head activities-were 
previously assigned to experienced teachers, with 
neither formal criteria for their assignment nor 
reimbursement for the time spent on them. 
Program participants report sharing of these 
duties and responsibilities. Said a principal from 
a career option program, "It has spread out the 
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responsibility, as well as the talents of the people 
in the program. There is no way that the three 
administrators could do the outstanding job all 
the consulting teachers did with the professional 
growth option." 

Teachers participating in career 
enhancement programs described increased 
support from their administrators for 
professional development activities. Participating 
teachers also noted increased respect of 
administrators and other teachers. Teachers 
whose participation brought financial gain 
reported a decrease in their resentment of 
teachers they perceived to be less capable, or less 
hardworking. Said one, "It has taken away 
resentment toward those who do not put in the 
same amount of time and energy but were paid 
the same as those who put in a great deal more 
time and effort." 

Career enhancement programs allow 
teachers to contribute to decisions concerning 
teacher employment and advancement. Teachers 
are accountable to each other, and have greatly 
increased opportunities for professional growth. 
These programs give opportunities to teachers to 
become leaders. 

Union representatives and administrators 
discussed the collaboration of district 
administrators, union representatives, and in 
some cases, school board members, to create 
these programs. Groups that often play 
antagonistic roles created programs that 



incorporated each group's goals and addressed 
each group's concerns. Thus, opposition to 
career enhancement activities was prevented. In 
addition, according to a program administmtor, 
credibility of the programs was increased and 
few respondents felt as if career enhancement 
was a choice made without consideration of their 
needs. 

Respondents called increased 
accountability for teachers an advantage. That 
is, teachers' actions and achievements in the 
classroom directly affected their opportunities for 
not only rewards, but in some cases, continued 
employment. This increased accountability 
resulted from increased wages for increased 
performance, or from the peer review process. 

When teachers are accountable to or 
evaluated by other teachers, they view the 
process to be more valid than when it is done as 
an administrative review. The evaluating teachers 
have not been "away from the classroom," as 
principals often are perceived to be. Additionally, 
accountability is to the profession, not just to a 
supervisor. A teacher from a career option 
program summarized, "It has broken down the 
barriers of 'you against us, and us against you.'" 
Another stated, "Peer reviewers absolutely do a 
better job than administrators of evaluating and 
mentoring teachers. There is no way any one 
administrator can spend the time to master the 
needs of all different grade levels and subject 
matters of his teachers." 



DISADVANTAGES 



No career enhancement program was without disadvantages. Disadvantages were related to 
personal conflict, competition, administrative problems, inadequate or subjective evaluation criteria, 
indifference of nonparticipants, and problems of time management 
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Conflict often resulted from the 
implementation of career enhancement programs. 
Educators from most districts responding to 
LOEO questions acknowledged several kinds of 
conflict as a disadvantage to their programs. 
These included nonparticipants' resentment of 
participants, dissension among teachers, changes 
in personal relationships, and jealousy. 



Several merit pay respondents suggested 
that getting higher scores on evaluations, rather 
than improving day-to-day teaching, had become 
the objective of some teachers. They indicated 
that although merit pay programs had produced 
change, some of the changes were "cosmetic," 
produced only to impress an evaluator. One 
teacher stated, "The only observable changes are 
cosmetic changes. Some people who ignored 
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bulletin boards or lesson plans may do them 
now. But their teaching is the same." 

Competition among teachers to achieve 
higher status in a program was identiHed as a 
disadvantage. One principal thought that this 
competition led to jealousy and personal feuds, 
and as a result, some teachers already performing 
at a low level now spent time looking for actions 
they could "tattle tale" about others, instead of 
improving their own teaching. 

A teacher also related that prior to his 
district's program, teachers would often 
collaborate on arranging field trips. However, 
the collaboration ended when one of them 
secretly arranged a field trip for her class and did 
not include the children of other classes. 
Coordinating field trips was one of many 
activities necessary to be considered for 
advancement, and this particular teacher hoped to 
score higher than her colleagues. 

Teachers who are not selected to receive 
rewards or recognition have feelings of jealousy 
toward those who are. One of the teachers 
whose performance has been recognized stated, 
"Some teachers are resentful that others get 
more. They think that all salaries should be 
higher. They think that if they just do their job, 
they should get paid more." A teacher's motives 
for change or improvement can be questioned. 
A teacher explained, "Within the staff, motives 
become suspect among peers. Any action makes 
me be suspected of 'brown-nosing.' I don’t like 
volunteering, because other teachers think I am 
only doing it to look good." 

Personal differences were mentioned as 
having been problematic in some programs. One 
consultant stated, "There is sometimes a 
personality conflict, which is not a system 
problem." Personal friendships can also hamper 
objectivity. A peer reviewer was assigned to 
assist a friend with whom he had worked for 
more than 15 years. He knew this person more 
than as a teacher recommended for intervention; 
he knew the teacher's family and he had spent 



leisure time with him. Although the peer 
assistance was successful and allowed the teacher 
to retain his job, the peer reviewer hopes he is 
never again assigned to monitor and assist a 
longtime friend. 

Actions of some administrators have 
detracted from potential program benefits. In 
one district, an administrator shared the results of 
teacher evaluations with all district personnel. 
Disparity between evaluations of coworkers who 
had perceived themselves to be equal in 
performance contributed to conflict and friction. 
One respondent said this action . . . 

[Cjaused a lot of dissension. We 
used to be a close-knit staff. But 
the first year of the program, the 
superintendent used the evaluations 
to rank-order the teachers, then 
announced the results. . . . Some 
really good teachers were quite 
angry that they were not rated as 
good as or better than their best 
friends. 

Some administrators have found it 
difficult to accept the role of peer review in the 
evaluation process. Differences of opinion 
between an administrator and a consulting 
teacher about what actions a particular intern 
should take to improve performance led to 
confusion on the part of the intern and conflict 
among all concerned. 

Indifference or lack of understanding 
by nonparticipants occurred in most programs. 
Teachers and principals not directly involved 
with career enhancement activities knew little 
about the programs. One principal returned a 
survey with a response of "Not involved, can't 
answer." In response to a survey in their own 
district, only about half of the district's teachers 
indicated interest in participating. In other 
districts, participants wished they had a better 
way to communicate about the program to 
nonparticipants. 
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Althou^ participants stated that their new 
responsibilities resulted in increased support from 
their administrators, nonparticipants did not share 
participating teachers' views of a diminished "us- 
against-them" battle. When teachers assumed 
responsibilities once considered uniquely admin- 
istrative, they reported occasionally being viewed 
by other teachers as part of "them" in this "us- 
against-them battle." A consulting teacher 
reported that experienced teachers equate referral 
to peer assistance "with the Hring squad. They 
see us [consultants] as negative people who 
affect their livelihood." 

Program administrators remarked that 
timing of dispersal of state funds is a 
disadvantage. One stated that often the district 
receives a check for a school year's program 
after payments to participants have been made. 
Another described the difficulty of receiving 
state money at the end of a school year for the 
preceding year's program. 

Participants in the merit pay program 
cited subjectivity of both the evaluation 
criteria and its application to pay raises as a 
program disadvantage. For example, how the 
principal decided on the distinction between a 
rating of "outstanding" and "satisfactory" was 
questioned, particularly by those who had not 
received the "outstanding" rating. Several 
participants, including those who had been 
ranked high, stated that the superintendent uses 
the rankings as one of several factors in pay 
increases, and that there is no way to ensure 
objectivity. 

Because of the relationship of the merit 
pay evaluation to the amount of a teacher's 
paycheck, it was difficult for principals to use 
the evaluations to help teachers improve. Said 
one principal, "We have four levels-- 
Unsatisfactory, Growth Area, Satisfactory, or 
Outstanding. I feel all teachers have growth 
areas, even the best ones. But if I cite a growth 
area, it takes away money firom the teacher." He 
explained that because his marking "growth area" 



decreased a teacher's paycheck, he was unlikely 
to do so. 

Disadvantages relating to time were 
referred to as respondents spoke specifically 
about peer review. One teacher expressed 
sentiments about the duration of the program 
when she stated, "In some cases eight months is 
not enough time to help new teachers, especially 
those who come from education colleges that 
have not taught them the basics they need to 
know. If the program were extended for two 
years, the new teacher would receive a greater 
benefit." A teacher in the shared governance 
program disagreed. In his district, an intern has 
two years of assistance. He said: 

One of the disadvantages of peer 
review is that if you have a really 
bad new teacher, that teacher has 
two years in the system when they 
really should be let go in the first 
year. I can understand that the 
union is interested in giving the 
person a chance to pull it together 
and turn around in the second year. 

But, if they don't, we have spent 
another year with kids not getting 
a good education. 

In one district, peer reviewers wished they 
had more time with individual interns. In that 
district, participants also mentioned scheduling 
difficulties. They commented that although a 
substitute was available for instances when they 
needed to be out of their classroom, they still 
had to prepare for the substitutes. A peer 
reviewer said, "We still struggle with how to 
have practicing teachers in the classroom 
teaching and also available to work with other 
professionals. Some of the lead teachers do not 
have enough time to assist the teachers." 

In other career enhancement programs, 
teachers often continue carrying a full teaching 
load, and add the responsibilities of their 
program activities. A career ladder teacher who 
creates an inservice activity for other teachers 
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does so in addition to preparing for her daily 
class work. In a district where peer reviewers do 
not have full-time classrooms, a respondent 
stated that the best lead teachers may not want to 
spend three years away from teaching children. 

A feature of one career option program 
was seen as an advantage by district admin- 
ishators, yet a disadvantage by some participants. 
In this district, an individual teacher's position 
within the career option program lasts no longer 
than three years. This ensures that they will 
return to the classroom. It also allows the costs 
of the program to remain stable. If the positions 
(which increase teacher salaries by 20% of their 
scheduled salaries) were for longer than three 
years, costs for paying the participants would 
increase as their scheduled salaries increase. 



However, some participants anticipate difriculty 
returning to the relative isolation of the 
classroom. They also expressed anxiety about 
acceptance by other (nonparticipating) teachers 
as they resumed classroom duties, because of the 
perception of some teachers that peer review 
duties were administrative rather than teaching. 

Respondents report an increase in 
evaluation-related work. One nrincipal said that 
the evaluations he was required to do for the 
district program made a "lot more work for me. 
I sp>end less time helping teachers, more time 
splitting hairs, trying to make sure that my 
written subjective evaluation can be turned into 
a supposedly objective one." 



CHAPTER IV 

CONCLUSIONS AND RECOMMENDATIONS 



LOEO found that career enhancement programs improved the overall 
quality of teachers in participating districts. They provided incentives for 
the best teachers to remain in the district’s classrooms, and increased the 
skills of participating teachers. Peer review assisted new teachers in 
developing classroom management techniques and allowed the least 
effective teachers to be removed from the classroom. Exhibit 6 summarizes 
our findings: 
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Decisions about teacher employment, advancement 
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Collaboration of teachers, administrators, union 
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Program Disadvantages 
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Getting high scores on the evaluation, instead of improving teaching, 
becomes the objective 
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Competition leads to personal feuds 
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Jealousy 
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Personal differences 
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Administrative problems. 
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Inadequate or subjective evaluation criteria 






" 1 


X 


Nonparticipants' indifference 




X 






Duration of program 










Scheduling difficulties 











* These were reported as part of the peer review program supported by matching funds. 



Conclusions 

Based upon interviews, surveys, and a 
review of the relevant literature, LOEO 
concludes that; 

Career enhancement pilot programs have 
flemonstrated success. 'Fhey benefited 
participiating teachers and their students. Each 
district designed its own career ladder, career 



option, shared governance, or merit pay program 
to meet its particular needs. 

The advantages of career enhancement 
programs benefited students and teachers. 
Career enhancement programs benefited students 
by contributing to teachers' use of new skills in 
the classroom, giving the best teachers an 
incentive to remain in the district, and helping 
ineffective teachers to leave the profession. 
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Career enhanc;iment programs also 
benefited teachers. High-performing teachers 
were rewarded with increased salaries, and in 
most cases, with increased responsibility, status, 
or respect. Teachers assumed leadership roles 
that were once solely administrative, such as 
evaluation and personnel decisions. Education 
reform efforts were enhanced by this 
professionalization of teachers' careers, which 
included increased accountability and collabor- 
ation of teachers, administrators, and the 
teachers' union. 

Most disadvantages resulted from the way 
the program was implemented in a specific 
district rather than inherent problems in the 
program itself. Disadvantages, such as 
personality clashes, administrative problems, and 
jealousy resulted from the interaction of 
individuals. Other disadvantages included 
subjectivity of evaluation criteria and shortage of 
time to devote to the program activities. 

The most promising strategies used by career 
enhancement programs are peer review and 
targeted staff development. The peer review 
process was identified as effectively removing 
the worst teachers from the classroom. Peer 
review also provides opportunities for teachers to 
evaluate and assist one another. Staff 
development that targets specific needs of a 
district's teachers allows them to acquire the 
skills they need to help their students learn. 

The merit pay program gave its better 
teachers an incentive to stay in the district, by 
rewarding them for good evaluation scores 
with an increase in pay. However, the merit 
pay pilot program did not include activities to 
improve the skills of participants or a way to 
remove the least effective teachers. Its criteria 
for pay increases were subjective, and results of 
participants' evaluations did not remain 
confidential. These attributes detracted from the 
value of this pilot program. 



Reconunendations 

LOEO recommends: 

>■ The General Assembly allow all districts 
who want career enhancement programs to 
compete for grant funds. The success of six 
different programs has shown that career 
enhancement can be advantageous to 
participating districts. There is no need to 
continue testing the concepts of career 
enhancement in pilot projects. Districts with 
existing career enhancement programs and 
districts that wish to develop new ones should 
compete for line item funds. 

ODE would develop guidelines for distrib- 
uting funds, based on financial need and the 
potential of proposed programs to affect both 
students and teachers. Districts would be 
strongly encouraged to include peer review 
and targeted staff development components in 
their plans. In addition, each plan would be 
required to include regular self-evaluation of 
both the program's implementation and 
results. 

>■ The Ohio Department of Education 
incorporate peer review and staff 
development targeting individual districts' 
needs into any statewide professional 
development plan it initiates. Those 
activities should continue where they exist, 
and be incorporated into future professional 
development activities of other districts. 

>■ School districts that wish to use merit pay 
programs address the design and imple- 
mentation flaws of the pilot program. 
Redesign should ensure that the program's 
evaluation and reward process is objective 
and there are program activities to help 
teachers improve. 
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APPENDIX B 
SURVEY QUESTIONS 



March 25, 1994 



1. Name 

Position 

School district 

Type of school (elementary, middle, high) 

Years in this position 

Years of teaching experience 

Have you participated in the Cdistrictl Program? Yes N o 

Phone number (for possible follow-up questions) 

2. a. Have changes in instructional performance resulted from the (district^ Program? If so, 

please describe them and give a speciHc example, 

b. Have changes in student activities or student performance resulted from the (district) 
Program? If so, please describe and give a speciGc example. 

3. a. Have there been advantages to implementing the (district) Program? If so, use 

examples to explain. 

b. Have there been disadvantages to implementing the (district) Program? If so, use 
examples to explain. 

4. Has the (district) Program affected you individually? If so, how? 

5. Has the (district) Program contributed to reform or restructuring efforts in your district? If 

so, how? 

6. a. Has peer review played a role in the (district) Program? If so, please describe it. 

b. What have been the advantages of peer review? 

c. What have been the disadvantages of peer review? 

d. What effect has peer review had on instructional performance? 

7. If there is anything else about the (district) Program or career enhancement programs in 
general that you would like LOEO to know, please tell us. 
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